fix: allow some flakiness for RPC tests by LesnyRumcajs · Pull Request #5973 · ChainSafe/forest

LesnyRumcajs · 2025-08-22T15:14:58Z

Summary of changes

Changes introduced in this pull request:

allow for some flakiness in the RPC tests, given they were failing repeatedly, most likely due to network conditions (but maybe the worker was overwhelmed due to FVM init)

Reference issue to close (if applicable)

Closes

Change checklist

I have performed a self-review of my own code,
I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
I have added tests that prove my fix is effective or that my feature works (if possible),
I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Summary by CodeRabbit

Tests
- Improved test reliability by adding automatic retries with exponential backoff and jitter; tuned retry count and delays and added clarifying comments.
Documentation
- Replaced the branded introduction with a concise deprecation notice guiding users to the new docs site (docs.forest.chainsafe.io).

coderabbitai · 2025-08-22T15:15:06Z

Walkthrough

Added retry configuration (exponential backoff, count=4, delay=5s, jitter=true) and explanatory comments to the rpc_test_ test override in .config/nextest.toml. Replaced the documentation top hero in documentation/src/introduction.md with a short deprecation notice linking to the new docs. No public API changes.

Changes

Cohort / File(s)	Summary
Nextest config update `.config/nextest.toml`	Added explanatory comments about network retries and jitter. Added `rpc_test_` override retries: `backoff = exponential`, `count = 4`, `delay = 5s`, `jitter = true`.
Docs deprecation notice `documentation/src/introduction.md`	Removed hero, badges, and branding; replaced with a concise “Documentation deprecated” notice linking to the new docs site.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

fix(test): increase nextest timeout for rpc snapshot tests #5945 — Modifies the same .config/nextest.toml rpc_test_ override (related changes to test timeouts/termination).
test: refactor rpc_regression_tests into individual tests #5930 — Updates .config/nextest.toml entries for RPC test overrides (related test config changes).

Suggested reviewers

hanabi1224
akaladarshi
sudo-shashank

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

MCP integration is disabled by default for public repositories
Jira integration is disabled by default for public repositories
Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 571d12e and 9974081.

📒 Files selected for processing (1)

documentation/src/introduction.md (0 hunks)

💤 Files with no reviewable changes (1)

documentation/src/introduction.md

✨ Finishing Touches

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch allow-flakiness-rpc-tests

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

Visit our Status Page to check the current availability of CodeRabbit.
Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (2)

.config/nextest.toml (2)
38-39: Comment clarity is good; consider linking to nextest docs instead of Wikipedia.

Nit: since this comment documents a nextest-specific feature, pointing to the nextest jitter docs provides a more durable reference for future maintainers. (nexte.st)

Apply within this range:
-# Jitter is enabled to avoid [thundering herd issues](https://en.wikipedia.org/wiki/Thundering_herd_problem).
+# Jitter is enabled to avoid thundering herd issues; see:
+# https://nexte.st/docs/features/retries/#adding-jitter
42-43: Improve retries config & verify override filter in .config/nextest.toml (lines 42–43)

The exponential‐backoff retry settings are valid, but you can make them more robust by bounding the backoff delay and adding a kill grace period. Additionally, verification scripts found no disabling flags and no tests matching your rpc_test_ filter—please confirm the override pattern matches your intended tests.

• File: .config/nextest.toml (lines 42–43)
• No usages of --retries or NEXTEST_RETRIES detected; per-test overrides won’t be inadvertently disabled.
• No Rust tests named rpc_test_* were found; ensure the filter test(rpc_test_) actually matches your test names.

Recommended diff:
-slow-timeout = { period = "60s", terminate-after = 2 }
-retries = { backoff = "exponential", count = 4, delay = "5s", jitter = true }
+slow-timeout = { period = "60s", terminate-after = 2, grace-period = "10s" }
+retries = { backoff = "exponential", count = 4, delay = "5s", max-delay = "30s", jitter = true }
Optional enhancements:
• Lock minimum Nextest version to ensure support for jitter and max-delay (added in 0.9.38):
nextest-version = { required = "0.9.38" }
• Surface retried tests in CI logs more prominently:
[profile.default]
status-level = "retry"

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

MCP integration is disabled by default for public repositories
Jira integration is disabled by default for public repositories
Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 5af1386 and 5c8bc90.

📒 Files selected for processing (1)

.config/nextest.toml (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📓 Common learnings

Learnt from: LesnyRumcajs
PR: ChainSafe/forest#5907
File: src/rpc/methods/state.rs:523-570
Timestamp: 2025-08-06T15:44:33.467Z
Learning: LesnyRumcajs prefers to rely on BufWriter's Drop implementation for automatic flushing rather than explicit flush() calls in Forest codebase.

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)

GitHub Check: cargo-publish-dry-run
GitHub Check: Build Ubuntu
GitHub Check: Build MacOS
GitHub Check: All lint checks
GitHub Check: tests
GitHub Check: tests-release
GitHub Check: Build forest binaries on Linux AMD64
GitHub Check: Analyze (go)
GitHub Check: Analyze (rust)

fix: allow some flakiness for RPC tests

5c8bc90

LesnyRumcajs requested a review from a team as a code owner August 22, 2025 15:14

LesnyRumcajs requested review from akaladarshi and hanabi1224 and removed request for a team August 22, 2025 15:14

coderabbitai Bot reviewed Aug 22, 2025

View reviewed changes

LesnyRumcajs added 2 commits August 22, 2025 17:23

back to prev timeout

571d12e

fix: remove deprecated docs info

9974081

LesnyRumcajs enabled auto-merge August 22, 2025 15:42

hanabi1224 approved these changes Aug 22, 2025

View reviewed changes

elmattic approved these changes Aug 22, 2025

View reviewed changes

LesnyRumcajs added this pull request to the merge queue Aug 22, 2025

Merged via the queue into main with commit 35d3d09 Aug 22, 2025
55 of 56 checks passed

LesnyRumcajs deleted the allow-flakiness-rpc-tests branch August 22, 2025 16:45

This was referenced Aug 25, 2025

test: add CI cache for rpc snapshot tests to mitigate flackyness issue #5978

Merged

Add forest api test-stateful subcommand #5836

Merged

coderabbitai Bot mentioned this pull request Feb 5, 2026

fix(ci): set CARGO_BUILD_JOBS=2 for unit tests #6538

Merged

6 tasks

coderabbitai Bot mentioned this pull request Mar 2, 2026

fix(test): print some logs in test_rpc_server for troubleshooting timeout issue #6670

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: allow some flakiness for RPC tests#5973

fix: allow some flakiness for RPC tests#5973
LesnyRumcajs merged 3 commits intomainfrom
allow-flakiness-rpc-tests

LesnyRumcajs commented Aug 22, 2025 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 22, 2025 •

edited

Loading

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Status, Documentation and Community

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

LesnyRumcajs commented Aug 22, 2025 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reference issue to close (if applicable)

Other information and links

Change checklist

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Status, Documentation and Community

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LesnyRumcajs commented Aug 22, 2025 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 22, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)